Riley, R. D., Cole, T. J., Deeks, J., Kirkham, J. J., Morris, J., Perera, R. et al. (2022). On the 12th Day of Christmas, a Statistician Sent to Me. BMJ, 379, e072883.
Statistical significance \(\neq\) clinical significance.
Conversely:
Absence of Evidence \(\neq\) Evidence of Absence.
Remember:
1 unit increase in the covariate has the same ‘effect’ on the outcome across the range of covariate values.
Linear splines - different linear functions across range of covariate values.
Regular polynomials - quadratics (\(x^2\)), cubics (\(x^3\)), etc (integer powers).
Fractional polynomials - fractional powers (e.g. \(x^{2/3}\))
Restricted cubic splines (RCS) - multiple regular cubic polynomials joined smoothly at ‘knots’.
Ratio of risk ratio’s = 0.76 (95% C.I. 0.49, 1.17; p = 0.2)
Another Absence of Evidence \(\neq\) Evidence of Absence problem.
Older women may still benefit from HRT but we can’t say with certainty - possible loss of power from splitting the data and reducing the sample size.
The current data don’t support that conclusion.
In meta-analyses, multiple smaller studies are combined to increase statistical power in order to estimate a treatment effect.
Ideally, all studies to be combined would be undertaken in the same way and to the same experiment protocols (e.g. study design, treatment regime, inclusion criteria, etc).
Differences between outcomes would then only be due to chance, and the studies would be considered homogenous.
But, this isn’t the real world. Studies are never done exactly the same.
Therefore, we expect some study heterogeneity arising from the variability in outcomes beyond that due to chance.
If heterogeneity is high, should the studies even be combined?
One (the most common?) measure of heterogeneity is \(I^2\).
\(I^2\) describes the percentage of variability in (treatment) effect estimates that is due to between study heterogeneity rather than chance.
It is a relative measure of heterogeneity and should NOT be interpreted as absolute.
0% - studies more homogenous (but the actual between-study variance may still be high - i.e. a small percentage of a large number may still be large).100% - studies more heterogenous (but the actual between-study variance may still be low - i.e. a large percentage of a small number may still be small).Meta-regression can be considered an extension of meta-analysis that combines, compares, and synthesizes research findings from multiple studies using regression analysis to adjust for the effects of available covariates on a response variable.
Covariates usually at the study level, rather than individual level (e.g. mean age, proportion men).
Usually underpowered.
Beware of aggregation bias -> ecological fallacy:
5% in exposed and 1% in unexposed, when in reality the risks are 50% and 10%.